Cathedrals and Bazaars
(for those of you who aren’t in the academic tech community, this is going to seem a bit inside-baseball…feel free to move right along)
Over the weekend, Bruce D’Arcus wrote a blog post about Zotero in which he said that “one gets the sense of a project at the portals of the cathedral gazing out at the bazaar, but not yet ready to step out the door.” This has sparked a bunch of discussion among us Zoterons over the past few days, and I wanted to distill a bit of it here.
From my particular perspective within the Zotero project, it’s less that we’re not ready to step out the door than the simple fact that we’ve been so focused on short-term goals that bigger conversations like the ones Bruce wants to open up have fallen by the wayside. Yes, we’ve been less than responsive to Bruce’s requests, but not out of any deep or unconscious desire to stay within the cathedral. In fact, I’d argue that the cathedral/bazaar dichotomy, while useful for highlighting different modes of work, is also rather reductionist, and can distill the gray areas of actual practice into black and white strawmen.
There are a number of models for open source projects; the one we’ve been trying to follow at Zotero is one of open participation, driven by participants themselves. At this point, if you want to participate, you can get full read access to our bug tracking and versioning system, and write access once you’ve proven yourself. While the dev list has been low-traffic (I wouldn’t go so far as to call it “dead”), the forums are extremely active, and we’re proud of a level of responsiveness by the whole team there that far surpasses expectations for an open source project. There are mechanisms for discussion, and we are deeply committed to F/OSS principles (along those lines, we’re going a step further and opening up our bug tracking system to full anonymous read access; Dan Stillman posted the details on the dev list yesterday). When we introduce something new, we see it as the beginning of a conversation, not the end of one: the Word plugin, for example, is a first draft, and we’d love to see someone elaborate on it. As for OpenOffice functionality, we haven’t had time yet to really start that conversation (either with words or code), but we’d love to see someone else do so (and in fact might be able to compensate them for their time).
On the Word plugin: it seems to have come out of nowhere mainly because it did come out of nowhere. One of our developers, who’s currently in college, needed to write a paper and came back with a working Word plugin. This wasn’t a top-down directive so much as him saying “Hey, this’d be cool,” and pulling it out of thin air (he’s actually got quite a reputation around CHNM for suddenly producing incredibly cool bits of code). As for the OpenOffice functionality, when Kari wrote “We do indeed have plans to add support for OpenOffice—it should happen sometime in 2007” the meaning behind that was “We’ve been talking to Bruce and others, and we’re hoping to work with them later in 2007 to get a similar plugin running for OpenOffice.” The statement was more opaque than it might have been, but it doesn’t belie a master plan evolving within CHNM to exclude OpenOffice folks (heck, we’d love it if someone else would step up and start building that functionality, rather than waiting for us to do so).
In the end, I think (and this is now just speaking for myself, rather than on behalf of Zotero as a whole) that the one thing that we could do better is make the process of feature development more transparent; it seems that Bruce’s main concern is that there are discussions happening and decisions being made behind closed doors, without real community participation. This is true on some levels; since most of us have offices along the same hall, we have a bunch of informal conversations that we don’t report back to the dev list or forums immediately (unlike most traditional open source projects, in which virtually all essential discussion happens in public, electronic fora). Our first concern has been simply getting software out the door that appeals both to our sense of good design as well as what our grant funders want, and in that rush we’ve often taken the route of quick conversations and decisions which can appear opaque from outside the CHNM offices. With that in mind, we’re going to try to do better, pushing those discussions out onto the dev list (where anyone else who’s concerned can join in while things are being figured out) when possible, and doing frequent reports on in-house projects and planned features out to the group…
Beyond that, however, there is an aspect by which Zotero will be always a bit cathedral-like; our current model is more like a service, where the core app is developed by a relatively small group of in-house or vetted devs, but with tons of hooks and an API enabling the flourishing of third-party utilities that extend it. We’ve created detailed developer documentation, including a sample utility, and we’ll be adding more to this as our APIs stabilize. Zotero has been built from Day 1 to be extensible in nearly every way possible. In short, less a massive cathedral than a small building in the middle of the bazaar…
January 30th, 2007 at 2:39 pm
Josh — good post. I just wanted to agree with your read on this point:
“In the end, I think (and this is now just speaking for myself, rather than on behalf of Zotero as a whole) that the one thing that we could do better is make the process of feature development more transparent; it seems that Bruce’s main concern is that there are discussions happening and decisions being made behind closed doors, without real community participation.”
Opening up the bug tracker was a good step.
But the next step is to actually use the dev list for conversation and community building, as opposed to the very occasional “OK, here’s what we’re doing” announcement vehicle.
I’ve been repeatedly saying that we collectively need to have a discussion about the intersections of data exchange model and CSL. When you guys add new reference types without some coherent plan of a) on what basis they get added, and b) how we’ll account for them in the formatting system (CSL), it makes things difficult for me. It is not that hard for you guys to post some thoughts on this to the dev list so that we’re all on the same page.
January 30th, 2007 at 6:49 pm
Josh, many thanks for your post, I appreciate your response! It’s truly amazing what the Zotero team has reached so far. That said, I do share Bruce’s sentiments.
As a third-party developer, I’m very much interested in helping out to build a thriving Zotero developer community, but what’s lacking right now is the big picture. What is your vision, what are your big plans for the future? I understand that it takes some extra effort (and risk) to share these ideas in the open, but it helps external developers to react accordingly and hopefully provide for helpful input.
In my opinion, the biggest thing about Zotero (besides Zotero’s translator capabilities) is it’s support for modern & upcoming bibliographic standards, such as Bruce’s CSL, or RDF, COinS and MODS XML. From the viewpoint of a third-party developer, these are the areas that I’d like to see developed further. Talk with us about your plans for extending these standards-based features (microformats, unAPI, RSS, OpenSearch, server/client API).
You said: “we’d love it if someone else would step up and start building that functionality, rather than waiting for us to do so”.
Developing for a foreign application always takes time and requires a sound understanding of its inner workings. By sharing your future plans early and letting third-party developers participate in your planning-stage discussions, you’ll actually increase chances that somebody else steps in and develops something before you’ve even found time to think of it yourself.
January 30th, 2007 at 9:49 pm
Thanks for addressing Bruce’s post. He spoke for a lot of us.
At this point, if you want to participate, you can get full read access to our bug tracking and versioning system, and write access once you’ve proven yourself.
I appreciate that you’ve decided to have anonymous Trac access. This will really help to see what you’re planning/doing & to provide helpful feedback to end-users in the forums.
We currently have access to the firefox extension, but the root is password protected & I don’t see any way to track your other projects. I would still like to see EVERYTHING you guys make in version control. I don’t believe the Word macro is in there yet.
If you guys have started your web-based collaboration tool, I hope that you give the URL for the repo (even before you release it). I know you haven’t started the OO.o extension (only because you’ve said ad much). I hope that, as soon as someone does start it, it will also be in subversion.
It is O.K. for a lone coder to take it upon himself to make a significant dent in a project (like the Word plugin). But that work should still become trackable by the rest of the “bazaar.”
the forums are extremely active, and we’re proud of a level of responsiveness by the whole team there that far surpasses expectations for an open source project.
The forums are EXCELLENT for end-users. Unfortunately, most developer questions do not get answered & I haven’t seen any that have provoked real discussion about design.
I am glad that you plan to push more of this design discussions onto the dev list.
January 30th, 2007 at 9:52 pm
“I reserve the right to clean up the code of comments.”
Ah—that’s why the HTML was stripped from my note. Sorry if it is more difficult to read without a clear use of quotes and the links to different svn directories. Let me know if something isn’t clear!
January 31st, 2007 at 9:43 am
Re. the word plugin in svn—Dan mentioned it is in the extension/trunk directroy:
https://www.zotero.org/svn/extension/trunk/
Guess I missed it when I was following the current branch.
January 31st, 2007 at 12:57 pm
Josh, you know I’m about as much a coder as I am a penguin, so I throw in my $.02 at risk of embarrassment. My believe is that you ought to stay in the cathedral for as long as you want. Opened Sauce has its place, but I think that there is too often a dogmatic presupposition by the developer set that it serves the user community to have as many cooks in the kitchen as possible, and at the cost of stability and usability. I’m absolutely sure you and your brethren are making wise choices, but the more “community involvement” ebbs towards “representative democracy” the more intolerable the software and the community tends to become.
In any case, FWIW, enter the bazaar with eyes wide open. I heart Zotero.
January 31st, 2007 at 12:59 pm
Tricky wordpress… That “absolutely” shouldn’t be crossed-out, and my heart should have asterisks around it. Have I even used an Internet before??
February 1st, 2007 at 9:15 pm
[…] Such promise generates excitement and early evangelists. One of those evangelists, however, frustrated with the pace and degree of responsiveness from the program, went public with a call to better embrace Raymond’s “bazaar” aspects of open source (see this reference and its comments). That, in turn, with its comments, spurred another response from one of the Zotero team leaders speaking unofficially called Cathedrals and Bazaars, also in reference to Raymond’s treatise. […]
February 7th, 2007 at 7:37 pm
Well said, Josh. I completely agree with you. I don’t have that much experience in Cathedrals but I do know how software gets developed – and it always seems to come back to reliance on those who are willing (and skilled enough) to actually do the work. That’s not to say that great ideas don’t occur to people who are incapable of joining the project as developers. Happens all the time – user feedback, suggestions, etc. I’d say the Zotero project has been much more open, much earlier than any grant-funded project I’m aware of. I suggest that once the development group has heard the same suggestion over and over it merits consideration – unless, of course, all the suggestions seem to come from the same person
February 12th, 2007 at 8:30 pm
I’m just curious to know if anyone outside the dev team has “proven themself” enough so that they have commit privs.
April 8th, 2007 at 9:09 am
@Ed: good question. I don’t know if there are any restrictions tied to their grant money, but it would be good for the long-term viabiility of the project to have outside committers. Imagine what would happen if for some reason the grant money dried up?
October 23rd, 2007 at 12:09 am
Cool article.
I hope you’ll link to my diary..
Ciao
October 31st, 2007 at 6:18 am
Sweet article.
I think you’ll read our blog..
Regards
November 6th, 2007 at 3:49 am
Hi this comment is delightful.
I like your site..
bye