Starting tonight I'll be at Gnomedex. Look for the guy with the camera attached...

Blogging has been light for a variety of reasons. One of the best has been David Hobby's Strobist Bootcamp. The first assignment was due yesterday. Here's the entry that I submitted.

I'm going to put all of my submissions (and outtakes) in this set.

You can see all the submission for the first assignment here.

I've been following the switch off the Mac conversations involving Mark Pilgrim, Tim Bray, John Gruber, and others. The major theme is around being data portability / lock-in, with open source applications being a minor theme. Like Mark, I've been through the data locked in proprietary application jail, both on the Mac and Windows. I don't like it, it doesn't taste good, and it's a real problem.

The thing that's bothered me as I've been reading is the focus on the data lock-in problem as the only problem. I expect my computer to help me work better and faster, and for that I need good applications, and some of the applications are one of a kind or in a category for which there are no standardized data formats. I'm just not going to stop using those apps and start managing that data as text files in Emacs. Sometimes in order to support the application, you need a richer file format. I don't think that is evil. But I agree that for common data types, Linux is ahead here.

I've been using Ubuntu since early 2005, and I've used Debian for many more years prior to that. When my Powerbook was stolen earlier this year, I switched to using Ubuntu as my desktop productivity environment (I was already using it as my primary development environment -- the Ubuntu box was the fastest in the house, at least until the Core Duo machines showed up). As I wrote previously, I just couldn't stomach it. I had access to most of my critical data, but I felt that I was significantly less productive on the Ubuntu box. Part of that was because I didn't attempt to port some of my productivity infrastructure to Ubuntu, but part of it was also the lack of applications and application integration infrastructure. I'm not averse to switching platforms. I went from Apple II to IBM PC to Classic Mac to Windows NT (while I was working at Apple, no less) and most recently to Mac OS X. I am averse to taking a major productivity hit, which is what happened for the months that I was on Ubuntu.

The fact that the operating system and (almost) all the apps that I was using on Ubuntu were open source didn't really make much of a difference. The last thing that I want to do when I am pressed for time is fix problems with my desktop apps, especially when there are superior alternatives. In theory, I would prefer my desktop apps to be open source, but unless I get really annoyed, it's unlikely that I am personally going to fix any problems with those apps. Many of the problems that I have with apps are issues that affect a minority of users, and for that reason, Apple, or any large vendor would prioritize those bugs way down. It's the same issue with long standing bugs with the JDK which Sun prioritizes way down. In either case, open sourcing the software would allow people outside the vendors to address the bugs. My experience on the Mac is that my problems getting bugs fixed is a large vendor problem. I use a lot of applications that are written by small independent developers or software houses. In general, I've found these developers to be as responsive or in some cases (Brent Simmons) more responsive than your typical open source project. It also is conveniently the case that the small developers are the more innovative ones. No, the apps aren't open source, but I get some (not all) of the benefits that I get from open source software.

What would it take to get me to switch to Linux/Ubuntu?

Fix the fonts - I run my monitors at microscopic. I was literally getting headaches from trying to use Ubuntu at the same resolutions.

I work on a laptop, with an external display. Undocking from the external display is a pain because all the windows move into stupid places. Same thing happens when I hook back up. Make the machine do something intelligent when this sort of thing happens. (No, OS X doesn't do this - but you're trying to get me to switch, right?)

Give me application integration/extension capability. Give me scriptable applications. Unix has plenty of scripting languages, but it has very few applications whose internal functionality can be accessed via those languages. I create custom workflows by using AppleScript and/or python to glue together GUI and command line applications to get the job done. This scripting capability allows me to use any application as a platform for developing my own custom applications using scripting. Also, OS X has services which can integrate into any application that uses the Cocoa frameworks. This means any Cocoa application can spell check or summarize text or look up the selection in the dictionary.

Color management. I never cared about this until I got serious about photography. Now I care about it a lot.

A system wide notification manager like Growl (I suspect this might already be in there). Make sure that those scripts can use it.

Working bluetooth synchronization between my phone and apps. When I can sync my address book and calendar to my phone and someone has written Sailing Clicker for Linux, you're done.

Lastly, give me good apps:

• Quicksilver - it works so well because apps are scriptable
• Aperture or Lightroom
• Equivalents to Mail Act-On!, MailTags - or a just plain better mail reader
• Good outlining and mindmapping
• An RSS Aggregator which provides a river of news style UI and can sync to a web based aggregator as well
• Ecto (none of the blog clients worked out of the box with pyblosxom) - integrated with the newsreader, so I can respond fast
• Rich del.icio.us client - integrated with the newsreader - so I can slam stuff into del.icio.us without messing up my browser state
• An IRC client with Snak style window/pane management
• Proxi - a good UI for taking actions based on system/hardware events
• Spell check in system supplied text fields
• A personal finance application that talks to banks and brokerages
• A small business accounting program that talks to the IRS directly
• Video IM
• Decent speech synthesis

I consider Thomas Hawk to be one of the superstars in the Flickr's community. He shoots prolifically, and almost every day I see a great shot of his go by. He's also very active in the Flickr on-line and offline communities. Thomas has been very vocal and hands on about photographer's rights. On top of all that, he also maintains a great blog where he talks about photography and technology in general. So when I read that he was going to Work for Zooomr, I paid attention. A lot of attention.

As I wrote previously, the thing that make Flickr sticky for me is the community of people. I have built up enough of a base there that I'm unlikely to switch to another photo-sharing site just for features. But if a bunch of people that are part of my personal community started hopping over to another site, then that would get my attention. The blogosphere is already aflame with the conversation about the technical end of this: should there be wholesale import/export API's or tools, who should provide them, and so forth. That's all fine, and I'm not worried about any of that, because in the end, any system that provides an API will end up having a way to do import/export. The more interesting question is, what could cause a sizable portion of the community to make the jump to a new system.

I think it's going to be interesting to see how this plays out, because Thomas has said he is going to keep on going at Flickr, so for the moment, there isn't going to be a break in the network. But if the day comes that Thomas switches to Zooomr as his primary, that's going to be very interesting. When companies try to build a product or service around a community, prominent members of that community are going to have more and more of an impact.

I wonder whether over time, we'll see further and more crisp stratification of the photo sharing market: Yahoo Photos for families, Flickr for people who want to learn and get serious about photography, Zooomr for the really hard core photographers... I'm not making a prediction about either niches or residents of those niches, but it is interesting to think about.

Well, not really, but maybe I had Scoble and Podtech worried for a nanosecond.

Rich Bowen and David Reid have started up the "Feathercast", an unofficial podcast featuring interviews with people from around the Apache Software Foundation.

Guys, a request -- better filenames!

Hanna Wallach and Chris Ball convinced the GNOME Foundation Board to fund some summer project slots for women, using the money that GNOME received from Google for participating in the Google Summer of Code.

If you know young women who are still available for some cool summer hacking, pass the word along!

Last week Jon Udell wrote a post titled Open source education describing how the transparency of open source projects can be a great value for people who are various degrees of learning about the field of software development.

Open source software development, to a degree unmatched by any other modern profession, offers apprentices the opportunity to watch journeymen and masters at work, to interact with them, and to learn how they think, work, succeed, and fail. Transparency and accountability govern not only the production of source code but also the companion processes of design, specification, testing, maintenance, and evaluation.

As the parent of small children, I would live it if kids are able to gain this type of insight about the fields that the choose to enter, before they enter them. Modern society is increasing the pressure on young people to make up their minds about what they want to do earlier and earlier in life. It (now) seems insane to me to slot young people into careers so early in life. I am fortunate that I am able to work in a field that I really enjoy, but I didn't really have a good idea of what it would be like to work in software before I got there. Judging from the number of people that seem to merely tolerate, or even hate their job, I'd say we're in need of a better way of helping young people figure out what the want to do when they grow up.

David Hobby has posted the first of the assignments for his Strobist Boot Camp. I'm definitely going to try to complete as many of the assignments as I can. This will be the first time that I've ever taken pictures that I intend to have critiqued, and I'm feeling a bit self conscious about that aspect. However, I know from experience with open source software that good critique is important to growing as a photographer.

The Strobist stuff seems to be pretty popular. I've seen David's blog cited in the DPReview.com and Fred Miranda forums. Also, David's publication of the "Starving Student" Off Camera Light Kit has single handedly wiped out the stock of Westcott Foldable umbrellas at a number of major photo shops. I know, because I just got a bunch of stuff from the kit, with the major exception of the umbrella, which is going to make the first assignment kind of tricky.

Here's a shot that I took while goofiing around with the kit with the kids last night.

Mike Arrington wrote about the Yahoo Photos Beta Launch:

Yahoo Photos is the largest online photo site with nearly 31 million unique monthly visitors (compare to Flickr’s 16.5 million). Flickr is for the early adopters. Yahoo photos used to be for the masses, but a few early adopters may be giving it a try, too.

I know that Mike is writing from the perspective of looking at the business and the features, which is perfectly fine thing to do. But the thing that is going to keep me on Flickr is not the features (although I'd like to have them), it's the community of people on Flickr. I've spent time on Flickr discovering people whose work I admire and follow, and Flickr makes it easy to do this. There are groups of people with similar interests, that I can learn from and share with. Groups like the one for the Strobist blog. For me, the Flickr on-line community has also crossed over into the real world. I've been to meetups in San Francisco and Seattle, so the connection is starting to be physical as well as virtual. I don't know if that's important to Yahoo or Flickr from a business point of view, but I know that it sure is important to me.

I'm sure that Yahoo Photos has the functionality to do all of these things. But I don't know if the people that I know from Flickr are going to be there. There are some attractive features on the new Yahoo Photos (I'm not in the beta yet, so I'm basing this on Arrington's piece), and I think that there might be some reasons that I would put some photographs up there. But the tie to Flickr is the community of people, and that's going to be the determining factor in where the majority of my attention goes.

It may be true that Flickr is/was for the early adoptors. But I also think that it's for the enthusiasts.

Mitchell Baker wrote an interesting post about The "community" and decision-making. She finishes with the important point:

Other projects may invoke the authority of the ultimate decision maker more or less often. The important point is that many open source projects incorporate clear decision-making into their processes.

There's some worthwhile perspectives on leadership in the comments.

Back in April Henri Yandell wrote an interesting post about the ASF's community mantra titled "How we lie to ourselves". In that post he contrasted "despot" driven projects and community driven projects, and he used Codehaus as an example of an organization that focuses on despot driven projects. Of course, he could have equally well used Linux, Python, or any of a number of fine open source projects as his example.

The term "despot" (or to use the Python community's label, "Benevolent Dictator For Life") inserts a lot of unhelpful color into this particular discussion. It is very rare to find a human endeavor in which no one displays that quality known as leadership. Sometimes it takes on the form of articulating a vision. Other times it takes the form of doing whatever needs to be done. In other settings it means stepping in to cool a raging discussion between project participants. There are various kinds of leadership, and all of them are necessary in a project of any size. As Henri points out, even ASF projects contain people who act like leaders - title or not.

I like to think of the two viewpoints this way: If you want to look at leadership as giving someone authority, then there are two kinds of authority. There's positional authority. That's where you give someone a title and then people follow the instructions of the person with the title. But there's another kind of authority, which is much more powerful (at least I think so). Call this relational authority, the authority that stems from the relationship that you have with another person. We all have (I hope) people that we trust enough that we let them "tell us what to do". We've given those people a place of "authority" in some sphere of our lives. In the "despotic" projects that I've observed up close, in every single one of them, the despot had gained significant relational authority, in addition to whatever their supposed title conferred.

A few more thoughts from "The Success of Open Source":

On the nature of open source leadership:

While leaders of other large projects have different personality traits, they do tend to share an attitude that underemphasizes their own individual importance in the process. And they share, more importantly, a commitment to invest meaningful effort over time in justifying decisions, documenting the reasons for design choices and code changes in the language of technical rationality that is the currency for this community

Because anyone can join and leave at any time, the leader is in a sense more dependent on followers than the other way around. The core question then becomes, How can a leader fail his followers? The simple answer, which captures the unusual power dynamic, is "lack of responsiveness to those led."

On the nature of the core freedom of open source (which is actually a freedom of community as opposed to property):

Albert Hirschman's classic argument from Exit, Voice, and Loyalty is a useful metaphor here. While the open source process certainly encourages the use of voice, open source licenses are design explicitly to empower exit rather than loyalty as the alternative when voice fails. The core freedom in free software is precisely and explicitly the right to fork.

In other words, positional authority without relational authority is moot, and the community is always free to vote with its feet.

I finished Steven Weber's "The Success of Open Source" a while ago, but it's taken me a while to make my note-taking pass, and even more time to figure out how to get a decent post out of my notes. Getting this post out has been holding me back from really digging into The Wealth of Networks, so here goes. Weber looks at the phenomenon of open source through the lens of political economy. The major questions from this view point are about motivation of individuals, coordination, and complexity.

One of the interesting conceptual handles that I took out of the book was Conway's law:

In 1968, Melvin Conway made an argument about this relationship that software developers now call Conway's Law. This is the idea that the structure of the (technical) system mirrors the structure of the organization that developed it. According to Conway, it is first and foremost a consequence of the communication needs of the people building the system. The computer scientist David Parnas takes this argument a step further by defining a software module as "a responsibility assignment" for a piece of an organization first, and only secondarily as a subprogram.

The open source process relies on a foundational consensus around technical rationality -- that is its stake driven firmly in the ground. It is actively evolving organizational structures around that stack. Thus open source developers really are forced to live by the lessons of Conway's Law. The architecture of the technical system drives the organization, rather than the other way around.

The reason Conway's Law actually works in favor of the open source process is that early architectural formulations are approximations are are always unstable. The formal hierarchical organizations that enclose proprietary software development are resistant to change and often end up driving the technical architecture (the dysfunctional flip side of Conway's Law). In the ideal case the organization should follow this organization rather than lead it.

Conway's Law definitely provides some more food for thought when look at the ASF's community is more important than code mantra.

A key notion in the "commons movement" is that of non-rival goods. Weber takes this a few steps further and gives us another term, "antirival" goods:

I believe the solution to this puzzle lies in pushing the concept of nonrivalness one step further. Software in many circumstances is more than simply nonrival. Operating systems like Linux in particular, and most software in general, actually are subject to positive network externalities. Call it a network good, or an antirival good (an awkward, but nicely descriptive term). In simpler language, it means that the value of a piece of software to any user increases as more people use the software on their machines and in their particular settings

The point is that open source software is not simply a nonrival good in the sense that it can tolerate free riding without reducing the stock of the good for contributors. It is actually antirival in the sense that the system as a whole postively benefits from free riders.

The twist is this: Under conditions of anti-rivalness, as the size of the Internet-connected group increases, and there is a heterogeneous distribution of motivations with people who have a high level of interest and some resources to invest, then the large group is more likely, all things being equal, to provide the good than is a small group

I think that the issues of governance and culture are going to become more important as familiarity with open source practices continues to increase both in software and other fields.

The Internet does not solve foundational problems of organizing complexity. It does not create working divisions of labor. Reducing or even removing the costs of geographically widespread and time-dependent collaboration is important, but that effort still leaves other collaboration costs unsettled -- decision making, human emotion, resolution of technical uncertainties, and so on. In principle, the Internet can increase these difficulties because it can be set up (and in the open source process, is set up) to be non-excludable

When I use the term "governance" in this discussion, I am using it in the way it is used in international relations. In that context "governance" is not government, it is typically not authoritative, and in fact it is not about governing in a traditional sense as much as it is about setting parameters for voluntary relationships among autonomous parties. Given that perspective, the central question becomes, How has the open source process used technology along with both new- and old-style institutions of governance to manage complexity among a geographically dispersed community not subject to hierarchical control?

Weber uses developments in modern religious communities as another example of reversing power dynamics in communities. I've observed this myself in some (but not all) religious communities.

More generally, there has been a dramatic shift in what it means to be a leader in a religious community. Change the foundations of property, and you change the network of relationships that radiate outward from that which is owned, in fundamental and unexpected ways.

I will come back to this point later when I discuss power and communities; but for the moment, recognize an interesting corollary. In nonauthoritative settings -- that is, in relationships primarily characterized as bargaining relationships -- power derives in large part from asymmetrical inter dependence. Put simply, the less dependent party to the relationship (the one who would be harmed least by a rupture) is the more powerful. In the kinds of modern religious communities I am talking about here, it is the leader who is dependent on the followers more than the other way around. This dependence changes the dynamics of power. Leaders of open source projects understand the consequences intuitively -- the primary route to failure for them is to be unresponsive to their followers. In Hirschman's terms, this is a community that empowers and positively legitimates exit while facilitating voice.

There were several interesting points around open source and innovation:

Innovation starts from the edge and migrates to the center as it proves itself -- there is a meritocracy of ideas as well, at least ideally.

End-to-end innovation goes a step beyond simply reduced transaction costs. It enables parallel processing of a complex task in a way that is not only geographically dispersed but also functionally dispersed. End-to-end architecture takes away the central decision maker in the sense that no one is telling anyone what to do or what not to do. This is the essence of distributed innovation, not just a division of labor. There are no weak links in the chain because there is, in a real sense, no chain. Innovation is incentivized and emerges at the edges; it enters the network independently; and it gets incorporated into more complex systems when and if it improves the performance of the whole.

This is of course an idealized version of the story. An end-to-end technological architecture may indeed be truly neutral, but technology does not exist on its own. The political and organizational interfaces that touch the technology and link it to human action typically are biased.

The underlying problem here is to specify general organizational principles for distributed innovation, a problem solving algorithm that works. The open source process suggests that there are four principles:

• Empower people to experiment
• Enable bits of information to find each other
• Structure information so that it can recombine with other pieces of information
• Create a governance system that sustains this process

The search problem of local vs global maxima applied to the innovator's dilemma at the level of individuals.

Distributed innovation systems like open source have a simple advantage here that helps compensate for the inherent tendency to stick to debugging an existing architecture. Any single open source developer might suffer from individual versions of the innovator's dilemma in her own decision making about where to invest her time, but the decision as a whole remains disaggregated among the community. ... Ultimately the freedom to fork code distributes much more widely the ability to make different bets on the exploration-exploitation trade-off, and that raises the probability that on aggregate some decisions will pay off. Put simply, end-to-end innovation systems depend on a market logic of disaggregated decisions rather than on a central intelligence to manage the innovator's dilemma.

The challenges facing the application of open source techniques to areas like user interface design.

But it is possible that end-to-end innovations systems will generally underperform in these specific kinds of user interface settings, for two related reasons. Some programmers believe that understanding direct user experience requires extensive physical interaction -- that is focus groups, not internet mailing lists. The more general point here is that design for components like user interfaces still rests on the transmission between individuals of a great deal of tacit information. This requirement makes it difficult to modularize and difficult to develop in parallel distributed settings. End-to-end innovation systems lack a good mechanism for getting at the underlying concepts of tasks like this in novel ways. That may change as advances in information processing move more human experiences out of the tacit realm and into communicable forms, but for now this seems a notable constraint.

Standards and open source: Open source aids standardization preventing the gaming of standards as described in Information Rules: A Strategic Guide to the Network Economy. I wish that open source was doing better at benefitting standardization as a source of unencumbered implementation experience.

In other words, releasing source code is a credible commitment against the possibility of using a standard for technological lock-in and hold up. Proprietary standards cannot make this commitment in the same way.

The success of open source is introducing many kinds of tensions between markets and communities. Weber has an interesting idea for looking at the needs of markets and communities:

In this kind of discourse, the failure of a community to take shape because of high transaction costs is just as much a market failure as is the nonevent of a contract to exchange money for widgets that would make both sides better off but does not happen because the widget owner can't find the buyer. The perfection of markets and the realization of potential communities are theoretically identical.

Weber believes that the next steps in research on open source have a lot less to do with transaction cost economics:

I do not have a full-fledged model to propose at this point, but I suspect any model will have to focus on at least two factors that transaction cost economics does not emphasize. The first is the relationship between problem solving innovation and the location of tacit information, information that is not easily translated into communicable form. ... The second is the importance of principles and values, in contrast to efficiency as an organizing concept.

A few days ago I commented on the difference between projects using open source methodologies and those which are loosely inspired by open source methodologies. I emphasized the governance end (meritocracy vs democracy). Weber also adds the property regime.

Many of these projects gain their ideological inspiration from the open source process and tap into some of the same motivations. Many are interesting experiments in using Internet technologies to organize volunteer efforts and affinity groups. But in many instances, these projects are not organized around the property regime that makes the open source process distinctive.

As as my personal history goes (all the way back to the Apple II), the entire category of "thinking tools" for computers originated with Dave Winer's ThinkTank outline processor and it's marvelous successor MORE. I use OmniOutliner on the Mac, and while it has features that ThinkTank and More never did, it just doesn't feel as natural as MORE. Then again, you're reading a guy who has over 20 years of Emacs muscle memory jammed in there too. Good as outlines are (and they are very good), sometimes you need something different. I've found that drawing a mind map is a good way to help me split out issues when I don't have a clear conceptual picture of what's going on. The big problem with mind maps is that they are hard to draw and maintain, especially when you are splitting nodes and moving them around. It's a total loss on paper -- it's precisely the thing that computers are good for. For a while I was using FreeMind, which is open source, but the interface left a lot to be desired.

Yesterday, MindJet launched the Macintosh version of their MindManager application for mind mapping. I've been using it as a beta tester for several months (Disclosure: I got a free copy of MindManager as a result), and I've been putting it to work generating maps of various kinds. I'm very happy with it -- the interface is reasonable, and I now have no qualms about creating a mind map if I think it would be remotely helpful for a problem that I am looking at. There is file compatibility with the Windows version of MindManager - I verified this by opening some maps that were created at Seattle Mind Camp 2.0. If you're looking for a mind mapping program, and you are picky about the interface, I think that you should download the free trial of MindManager for Mac and see for yourself.